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estimates of cosmological parameters. We propose an iterative Monte-Carlo simulation 
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upon fits of the dark energy equation of state. This approach is especially useful to 
track the impact of non-Gaussian, correlated effects, e.g. reddening correction errors, 
brightness evolution of the supernovae, K-corrections, gravitational lensing, etc. While 
the tool is primarily aimed for studies and optimization of future instruments, we use 
the "Gold" data-set in Riess et al. (2007) to show examples of potential systematic 
uncertainties that could exceed the quoted statistical uncertainties. 
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1. Introduction 

The direct evidence for dark energy, which were obtained through observations of Type 
la supernovae (SN la) in the 1990s, marked the beginning of a new era in observational 
cosmology [H [21 [3J HI [5] . Significantly improved SN la data sets have since then been 
reported in e.g. [61 [7J [8j [9] . These data sets have been followed by the impressive results 
from the ongoing dedicated SN la surveys: SNLS [TDJ and ESSENCE [11] . An important 
recent compilation of published SNe la, including e.g. the supernovae from the first year 
of SNLS, but also adding several high redshift [z > 1) SNe la from the GOODS survey, 
is reported in [12]. 

As the SN la sample sizes keep growing at an increased rate, the relative importance 
of statistical uncertainties is steadily decreasing, which brings systematic uncertainties 
to focus. It is thus of key importance to estimate the impact of systematic uncertainties 
as we enter the era of "precision cosmology" . 

In this work we propose a Monte Carlo simulation approach to quantify the 
propagated effect of known (or suspected) systematic effects in supernova cosmology 
The emphasis here is to quantify the degradation of the precision to estimate 
cosmological parameters due to systematic errors, possibly correlated in arbitrary 
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groupings of supernovae: in redshift bins, by locations in the sky, observational 
instruments, time of observations, etc. 

The current study includes investigations of sources of error from: uncertainties 
in the extinction by dust, K-corrections, redshift uncertainties, gravitational lensing, 
Malmquist bias, possible brightness evolution of the "calibrated" standard candle (e.g. 
metallicity effects), spectral template differences, and miss-calibrations between the low 
and the high redshift supernova sample. Systematic effects caused by bulk motion have 
been specifically targeted in a number of recent studies, e.g. [13], and were not specifically 
included in this study. However, some general systematic effects, e.g. drift of estimated 
peak brightness at low-z, would mimic the main effect expected from peculiar velocities. 

The proposed method, implemented in the SMOCK package and presented in §|3l 
allows us to quantify the sensitivity of cosmological results, obtained from real or 
simulated data, to specific (multiple) systematic effects. For example, we can quantify at 
which level a systematic effect gives rise to a bias exceeding the statistical uncertainty. 
The tool could hence be used to define requirements of any future mission aimed at 
improving our knowledge of the dark sector using SNe la. As an example of how this 
technique can be used we have performed a dedicated study of the "Gold" sample 
presented in Reference |12j . This study demonstrates how cosmological parameter fits 
are affected by a number of different systematic errors. 

In £j2]the model of the Universe which we have used and the cosmological parameters 
are introduced. Our methodology for studying the effects of systematic uncertainties 
is described in ^31 In §H] we show how the changes in the cosmological results due to 
systematic uncertainties can be quantified and the trends parametrized. A number of 
different systematic effects, their implementation, and their effects on the Gold set are 
the topics of §0 Some observations regarding systematic effects and the Gold set are 
presented in §[6j The paper is concluded by some implications for current and future 
SN la surveys in £J71 

Throughout the paper units are used where the speed of light is unity, i.e. c = 1. 
2. SN la magnitudes and cosmological parameters 

In this paper we investigate the accuracy to which the cosmological parameters 9 = 
(QM,flx,wo,w a ) can be determined. These parameters describe the matter density, 
Qm, the dark energy density, Qx, and the dark energy equation of state parameter, 
which is parametrized by w(z) = wq + w a z/(l + z). 

The apparent magnitude in an arbitray observer filter Y, my, of a SN la at redshift 
z is given by: 



m y (0, Mx, z) =M X + 5 log 10 d' L (6,z) + K XY (z) + A XY (z), 



(1) 



where 



M x = 25 + M x -51og 10 # , 



(2) 
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is a "nuisance" parameter depending on the Hubble parameter, H , and the brightness- 
shape corrected absolute magnitude, M x , of the supernova in restframe broadband 
filter X. The Hq- independent luminosity distance is denoted by d' L = H^d^. Kxy 
corresponds to the K-corrections involved in the transformation between magnitudes 
in the restframe X-band and the observed magnitudes in the F-band, as defined in 
Reference [H]. Axy is a general color correction which could include, for example, 
corrections for host galaxy extinction in the restframe X-band (F-band in the observer 
frame) and intrinsic color dispersion. 

The if - m dependent luminosity distance d' L is given by 

f sin(v^n^J) Q k <0 

d' L =(i + *)< / n K = o , (3) 

[ 7 ^sinh( v / n^i) n k >o 

n K = 1 - M - ttx, (4) 

i (5) 



H'(z' 
H'{z) = H(z)/H 



f(z) = exp 



y/(i + zf n M + f( z ) n x + (i + zf n K , (6) 

, 1 + w(z') 



dz 



(7) 



1 + z' 

where we consider, as already noted, an equation of state parametrized by 

w(z) = w + w a z/(l + z). (8) 

We will also refer to simply 'to' when a constant (w a = 0) equation of state parameter 
for dark energy is considered, e.g. in the figures showing confidence level contours in the 
(Qm, iw) -plane. 

In this article we focus on the implications of systematic uncertainties on the 
estimation of the dark energy equation of state parameter. To obtain the results 
presented here, a flat universe (Qk — 0) was assumed together with constraints from 
the BAO peak in the SDSS sample of luminous red galaxies |15j . There the parameter 

was measured to great precision at z\ = 0.35: A = 0.469 ± 0.017. This was included as 
a prior. 

Moreover, when showing results for the (wq, io a )-plane we marginalize over the Qm 
parameter. The method, presented in the next section, can of course be applied to any 
other cosmological parameters using any combination of priors. 



3. Methodology 



The aim of this study is to investigate how different systematic effects influence the 
cosmological parameters obtained from fits on a set of SNe la. In order to take both 
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statistic errors and systematic effects into account, we use Monte Carlo simulations. 
The method, in short, consists of the following steps: 

(i) The original data-set, consisting (minimally) of supernova redshifts and peak 
magnitudes, is used as a starting pointjj]- Other properties, like excess color, can 
be included as well. A perturbed synthetic data-set is generated by randomizing 
these quantities for each supernova according to either measurement uncertainties 
and/or assumed dispersion in the original data-set and the systematic effect (s). 
This is done taking into account possible correlations between supernovae. 

(ii) Cosmological parameters, 9, are fitted to the perturbed data-set. 

(iii) The first two steps are repeated many times and the density of best fit values of 
cosmological parameters are used to trace out confidence level contours. From the 
relative location and shape of these contours it is possible to understand and to 
quantify the influence of systematic effects, both in terms of potential biases and 
enlarged confidence level contours. 

Let us now elaborate on and motivate these steps. 

The original data-set in step (i) could either be a simulated data-set, corresponding 
to a known cosmology, or a real data-set for which we would like to study the expected 
effects of systematic errors. 

When the perturbed synthetic data-sets are generated, all applicable uncertainties 
and biases are used to compute the supernova redshifts and peak magnitudes. Since 
many sources of error may be included in a single run, each contribution is co-added 
in the final perturbed synthetic supernova data. For example, the distance modulus to 
a simulated supernova can be computed in several steps, each adding a modification 
to the apparent magnitude. First, statistical uncertainties are taken into account: a 
value of the peak magnitude (e.g. in rest-frame 5-band) is drawn from a Gaussian 
distribution centered around the original supernova brightness and with standard 
deviation corresponding to the intrinsic brightness scatter together with (for real data) 
the measurement error added in quadrature. Thereafter, other perturbations are added. 
For example, a constant bias, mimicking a calibration error, could be added to all SNe la 
in a particular redshift bin. 

Color errors or spectral template changes, and the related reddening corrections, 
also result in offsets from the originally derived distance modulus. Redshift uncertainties 
are generated from the probability distribution, if given by photometric redshift error 
studies, or by a smaller Gaussian error, if spectroscopic. All of the above modifications 
can be applied to the complete data set or just a subset (e.g. only high redshift objects). 
If only the intrinsic scatter, the measurement error or the sum added in quadrature is 
used (and assumed to be Gaussian in magnitudes), the standard analysis of the data-set 
is retrieved. 

| Instead of these derived properties, the supernovae could be described by their light curves observed 
in different wavelength bands. So far, our code does not calculate errors from modifications of individual 
measurement points. 
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In this paper, such a data-set will be referred to as a "clean" data-set. Two 
advantages with this method are that uncertainties and biases can be non- Gaussian 
and correlated. 

Non-Gaussian error distributions can be studied since the measurements can be 
randomized according to any arbitrary distribution. Examples of such distributions are 
the dimming by dust and gravitational lensing magnification. Arbitrary correlations 
can be created through modifications of subsets of the data. 

Any uncertainties in priors used should also be incorporated according to this 
method. If the prior is Gaussian, a value of the prior is drawn from a Gaussian 
distribution for each iteration. 

In step (ii) the best fit cosmological parameters 9 for the synthetic data-set are 
found using a maximum likelihood technique. 

In this paper, a likelihood analysis taking only Gaussian errors into account is 
performed. To find the best fit cosmological parameters to the synthetic data-set, the 
negative likelihood is minimized using the Powell algorithm [16] implemented in the 
SNALYS fitting routine within the SNOC package pi]. 

In step (iii) we use the best fit values of 9 from a large number of realizations 
of synthetic data-sets to construct confidence level contours. The best fit parameters 
9 obtained for each synthetic data-set can hence be viewed upon as one possible 
"measurement" of the cosmological parameters including systematic uncertainties. 
The ensemble of best-fit values stemming from the iteration procedure maps out the 
confidence region of possible "measurements" given the data with systematic and 
statistical uncertainties. Figure [1] shows examples of the best fit values of w and 
Qm as well as wo and w a for 8000 perturbed "Gold" data-sets. In each case, the 
fit also includes a BAO prior from [15]. To obtain the confidence level contours in 
Figure [1] and subsequent figures we use the density of best fit values of the cosmological 
parameters 9. The contours corresponding to e.g. the 68.3% confidence level in the 
plane spanned by Qm and w (right panel in Figure [[]) and w and w a (left panel) 
are found by identifying the isocontours which enclose 68.3% of the points, We have 
checked that the confidence level contours obtained in this way agrees with contours 
from standard % 2 minimization, when the errors are Gaussian and uncorrelated. In the 
standard interpretation of confidence level contours, the 68.3% confidence level contour, 
for example, is the contour which fulfills x 2 — Xmm + 2.30. [JJ 

This iterative method was developed mainly in order to take correlated and non- 
Gaussian errors into account (e.g. from extinction and gravitational lensing). When 
using standard maximum likelihood techniques, all errors are required to be Gaussian 
and the correlation matrix has to be calculated and inverted for each systematic error. 

§ For figures displayed here we have used the elliptic contour that closest match the true isocontour. 
The two are generally similar and elliptic contours make comparisons clearer. 

|| Subtle differences between the two interpretations of confidence level could arrise. So far no deviations 
have been detected, but a more dedicated study might be of interest and is a possible topic for future 
SMOCK-related work. 



Quantifying the impact of systematic uncertainties in supernova cosmology 



6 



When dealing with large data-sets this inversion can be numerically cumbersum and 
extremelly time-consuming. Using the SMOCK method this is avoided - each iteration 
is treated as a single occurrence and only the best fit is calculated^ It should be noted 
that in this fit [step (ii)] we use the standard procedure of assuming all SNe la to be 
uncorrelated with Gaussian errors, but since a) only the best fit is calculated and b) a 
large number of individual fits are used the result will be more accurate and the process 
less demanding for large data-sets. 

The code developed to run the loop consisting of steps (i)-(iii), and in particular to 
generate simulated data-sets taking systematic effects into account in a flexible way, is 
collected in the SMOCK package. 
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Figure 1. Examples of the outcome of a SMOCK run. The points show the best 
fit values of 8000 simulated data sets similar to the Gold data set. The inner and 
outer contours, obtained from the density of points, correspond to 68.3% and 95.4% 
confidence level. Left and right panels show the best fit values in the (uio, u> a )-plane 
and (SIm, tf)-plane, respectively. 



3.1. An example: the Gold data-set 

As a test case, we have chosen to study the "Gold" sample presented in Reference |12j+l . 
The data-set was further divided into three primary redshift bins: i) 36 low-z SNe la 
(z < 0.1), ii) 90 intermediate redshift SNe la (0.1 < z < 0.7), and iii) 56 high-z 
SNe la [z > 0.7). This division in redshift space was done in order to study the impact 
from systematic effects applied at different redshifts. It should be emphasized that 
this division of supernovae is somewhat arbitrary and mostly done for demonstration 
purposes. However, the redshift bins broadly correspond to i) the near-by SN la sample 
used to anchor the Hubble diagram [TBI HHJ [20], ii) the ground based data from the 

% The covariance matrix for any error could in principle be calculated through the Monte Carlo 
simulated data-sets. 

+ From which we have used SN la redshifts, colors, magnitudes and corresponding errors. The colors 
were derived from the host galaxy extinction, Ay, through the assumption of E{B — V) = Ay/Rv 
where Ry = 3.1. 
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High-Z Team, the Supernova Cosmology Project and the SNLS, and iii) the HST data 
from the highest-z SNe la. 

4. Propagation of systematic effects 

Through repeated SMOCK runs using a range of errors of different magnitudes we are 
able to make comparisons between the impact on fitted cosmological parameters from 
different systematic effects and standard statistical uncertainties. Figure [2] shows an 
example of the parameter fits which would result if the peak brightness magnitudes of 
the high-z sample were positively biased. The solid contour shows the 68.3% confidence 
level contour for the unperturbed (clean) sample, which only suffers from statistical 
uncertainties. The other contours in the figure correspond to data sets affected by high 
redshift apparent magnitude bias in the range 0.01 to 0.15 mag. An archive consisting 
of figures of all systematic effects individually applied to each redshift bin in the Gold 
set can be found at the SMOCK webpage www.physto.se/~nordin/smock. 




Figure 2. Effect on cosmological results due to high-z SN la with biased magnitudes. 
Left and right panels show 68.3% confidence level contours in the (wo, w a ) -plane and 
(f^M, w)-plane, respectively. Solid (blue) contours correspond to the results of the 
clean run, which only take into account statistical uncertainties. The other contours 
show the effect of adding bias of 0.01, 0.03, 0.05, 0.07, 0.1, and 0.15 mag. Flat universe 
and BAO prior assumed. 



To further quantify and compare different systematics we have monitored the 
functional dependence of the fitted parameters 9 and some other indicators on the 
assumed systematics. In what follows we will refer to the shift of the best fitted values 
or its changing uncertainty due to increased systematic effects as the "evolution" of a 
parameter or indicator. In particular, we will exemplify the evolution of five parameters 
and indicators due to increasing levels of systematic uncertainties: 

(i) The evolution of (best fit) Om and w. 

(ii) The evolution of (best fit) w and w a . 
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(iii) The evolution of the size (area) of the confidence level contours^. 

(iv) The evolution of the total area. This is defined as the complete confidence region 
that would have to be used if the systematic error was suspected^] 

(v) The increased dispersion around the best fit model in the Hubble diagram. Possible 
systematic effects that increase the scatter above some limit would be considered 
unrealistic. 

We have parametrized the evolution of the five parameters and indicators listed 
above as a function of the magnitude of the systematic effects. The goodness of each 
parameterization is estimated through the residual scatter around the best fit of the 
evolution. Notice that these parameterizations depend on the data set used and the 
redshift binning chosen. In most parameterization linear with the size of the 

systematic effect yields a good fit. Where quadratic or exponential parameterizations 
have been used, this is noted. 

Uncertainties added to the SN la data affect the dispersion in the Hubble diagram. 
This provides a way of estimating whether systematic effects are realistic or not - a 
systematic effect introducing a dispersion larger than the observed one is unlikely. We 
have therefore also monitored the added dispersion on the Hubble diagram as systematic 
effects were introduced (i.e. increased residual scatter around the best fit cosmology). 
We find that constant bias often yield very small changes in the original dispersion, 
while added scatter give rise to increased dispersion, as expected. In general, the bias 
from systematic effects begin to dominate over statistical errors well before causing a 
noticeable increase in the residuals in the Hubble diagram. A typical example will be 
discussed below (see §5.4p . 

As an example of how the SMOCK technique could be used, we calculate the limits 
for when each individual systematic effect will dominate over the statistical errors for the 
Gold data-set. This was defined to occur either when the area of the 95.4% confidence 
level contour in the (J1 M , w)-plane or the (w , w a )-plane was doubled (compared to the 
clean fit) or when the best fit of Om or to or, in the other case, wq or w a was outside 
the 95.4% confidence level region of the clean fit. This might be called the point of 
"systematic-statistic equality", and is used by us to compare the relative importance of 
different systematic effects. These points are summarized in Table Q] (for the wq — w a 
parametrization) and Table [2] (for the Qm — w parametrization) . 

Figure [3] shows the evolution of the parameters and indicators described in Section H] 
for the Gold sample with added constant magnitude bias. FigureHJshows the evolution in 
the (fiftf, w) plane when the absolute magnitude of SN la evolves with redshift. Graphs 
showing the evolution of systematic errors for a larger range of simulated errors can 

* This area is always used in comparison to the area corresponding to the clean sample, 
jt This parameter will depend on the changes of the first three, as well as the relative "direction" of 
error propagation in the fitted parameter plane. Looking at Figure [U an uncertainty of up to 0.15 mag 
would mean that the total area covered by any ellipse plotted should be included in the total area. 
This area is quoted compared to the area corresponding to the clean sample. 
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Figure 3. Magnitude bias for high-z SN la (systematic offset for all SNe with z > 0.7). 
Horizontal lines in the top left and top right plot show the limits when the statistical 
68.3% and 95.4% confidence levels are reached by systematics. In the bottom left and 
bottom right plot the horizontal lines show when the area relative to the clean run 
is doubled. Vertical lines show at which error (if so) these limits are reached. This 
signifies either a bias creating an effect larger than the statistical uncertainty or a 
systematic uncertainty doubling the size of the regions (or a combination thereof). 

be found at www.physto.se/~nordin/smock. Some of the main results for the different 
systematics are presented below. 

5. Specific sources of systematic errors 

In the current study, we have concentrated on a limited number of tests for systematic 
effects. We emphasize that the SMOCK code is flexible enough to easily incorporate 
further effects. In this section we discuss motivations, implementations and general 
results of the systematic effects we have studied. 

The color correction Axy was modelled according to the Galactic extinction law 
in Reference |21| . with the parameter Ry = AyjEiB — V) specifing the wavelength 
dependence. If an average Milky Way value of Ry = 3.1 is assumed, it is possible 
to determine Ay (i.e. restframe l/-band absorption) through the measured supernova 
color excess, E(B — V). In studying systematic effects we are not concerned with the 
absolute value of the correction Axy, but in its change as any of the parameters Ry 
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Figure 4. Bias in the (fijw, )w-plane caused by evolution of peak brightness as a 
function of redshift. Horizontal lines in the two top plots show limits when statistical 
68.3% and 95.4% confidence levels are reached by systematics. In area plots horizontal 
lines show when the area relative to the clean run is doubled. Vertical lines show at 
which error (if so) these limits are reached. This signifies either an introduced bias 
creating an effect larger than the statistical uncertainty or an introduced uncertainty 
doubling the size of the regions (or a combination thereof). 



and E(B — V) change. 

A particular difficulty comes with the use of cross filter K-corrections, Kxy{z), and 
spectral templates. Applying accurate K-corrections is notoriously challenging since 
they rely on spectral templates from compilations of nearby supernovae. Obtaining 
a spectral sequence as a function of time for each high redshift SN la is practically 
impossible with current allocation of instruments for follow-up observations. Since the 
same spectral templates are often used, any errors in the templates would propagate 
through the K-corrections and give rise to correlated errors between supernovae at 
similar redshift s. 

We have included the effects of possible systematic effects due to K-corrections and 
color corrections in two ways: 

(i) When any of the parameters (e.g. z, E(B — V) or Ry), were perturbed both clean 
and perturbed corrections were calculated for each SN 180 and the difference was 



ff These perturbed corrections include integration of the filter with extincted and redshifted spectra. 
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Figure 5. Effects from dust extinction evolving with redshift parameterized as a linear 
modification (SR V — k * z, k systematic error) of local (Milky Way) values. Effects 
on wo — w a plane demonstrated. Horizontal lines in the two top plots show limits 
when statistical 68.3% and 95.4% confidence levels are reached by systematics. In 
area plots horizontal lines show when the area relative to the clean run is doubled. 
Vertical lines show at which error (if so) these limits are reached. This signifies 
either an introduced bias creating an effect larger than the statistical uncertainty or 
an introduced uncertainty doubling the size of the regions (or a combination thereof). 



used as an individual SN la magnitude modification. 

(ii) To study the effects from different spectral templates we compared the differences 
in cosmological results which occur for three different spectral templates used in 
the calculation of the K-corrections. 

In order to study variations with redshift, systematic effects were applied to 
individual redshift bins. It is important to notice that systematic errors, including 
K-corrections, usually do not add linearly. For realistic combinations of different errors, 
simulations combining all of these are necessary. A simple addition of errors due to 
different systematic effects might consequently not correspond to the true combined 
error. 

We have used standard Bessel filters for this study and unless otherwise specified a modified Nugent- 
type spectral template |28j . In each individual K-correction calculation, the filter most overlapping 
with the f?-filter redshifted by a factor (1 + z) is used. 
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Table 1. Points of systematic-statistic equality for different systematics. Parameters 
and indicators in the table are described in Section!?) All fits are linear unless otherwise 
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int 


26.9% 


28.0% 






Malmquist bias 


low 


71.2% 


75.7% 




50% 



a An exponential fit to the error level had to be used to reach good precision 
b A quadratic fit to the error level had to be used to reach good precision. 



5.1. Magnitude bias or dispersion 

Instrumentation errors, calibration errors, peculiar velocities, or bulk motion could in 
principle give rise to redshift dependent bias or increased dispersion in SN la magnitudes. 
We have investigated the effects of magnitude bias and dispersion for different redshifts. 
Figure [2] shows an example of the effect of magnitude bias for the high redshift bin. 
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Table 2. Points of systematic-statistic equality for different systematics. Parameters 
and indicators in the table are described in Section!?) All fits are linear unless otherwise 



noted. 


Error 


z-bin 




w 


Area 


Tot. Area 


Bicis 




0.103 


0.100 




0.277 


Bicis 


int 


0.181 


0.179 




0.198 


Bicis 


1UW 


070 


069 




1 98 


£L/y±J — V j Uldb 


high 


n 094 

U.UZ^l 


094 




U.UU i 


H/y_D — V J Dido 


1111 


u.uou 


037 




040 
u.u^u 


H/yJD — V J Dido 


low 


017 


U.Ul ( 




033 
u.uoo 


W( R V"\ QT»rpnH 
H/y JD — V J bpifcJclO 


high 






999 


914 

U. Zi L^± 


F,(R — V\ <mrpflH 

J-J {J-J V J OjJl CcXLl 


i lit 






380 


378 


R y") c-nrp?irl 








1 96 

W. -1 ZiU 


0.122 


l\/T £1 (T At7Y~ll 1 1 1 1 i"\Ti 
iVlCbX CVUlLLllUll 


■in 


n o&i 


080 






A/Tfi 1 mm i i Qt nitiQ 

IVlcMlll^J lllo L Uld-D 


n ion 
111^,11 


94 R% 


95 6% 

ZtJ . U /(J 






1\/T ci 1 mm lief nitic 
IVicMllliJ Lllo L UldiO 


int 


^0 Q°/n 


4Q 7°/n 






l\/T ci 1 mm lief nitic 
IVlellllliJ Lllo L Uldo 


low 


\JO. i /o 


(Sfi 9% 




fiQ A°7n 


i Ly UlcXD 


n l oh 
111^11 


4 


4 








1X1 L 


14 9 

-Lr±. Zi 


1 9 & 
± z . o 






A?t r ni CI Q 


low 


1 9 

_l_ . Z 


1 9 
1_ . z 






/-ii r DirA li i t i r~\n 


•ill 

till 




3 Q39 








•ill 

fill 






3 71 7 

O.I 11 


^ 489 




n l on 
111^11 






1U. U<JU 


11.482 


/?tt- QT*TT>Cirl 


int 






35 443 


36 061 


/it/- Q"nTPfin 


1 rwxi 

1UW 






7 44 S 


7 1 79 

1 . X 1 Zi 


Mag spread 


high 


0.961 


0.948 


2.869 


1.162 


Mag spread 


int 


2.805 


2.511 


10.656 


3.012 


Mag spread 


low 


0.698 


0.721 


0.803 


0.508 


Redshift bias 


high 


0.039 


0.039 






Rcdshift bias 


int 


0.085 


0.092 






Redshift bias 


low 


0.001 


0.001 






Photometric z 


all 


0.007 


0.007 


0.007 


0.005 


Redshift spread 


high 


0.644 


0.693 


0.396 


0.296 


Redshift spread 


int 


0.032 


0.032 


0.271 


0.042 


Redshift spread 


low 


0.007 


0.007 


0.008 


0.005 



Results An additional Gaussian dispersion (e.g. due to measurement errors and 
intrinsic dispersion) give rise to increased confidence level contours and do not introduce 
any bias, as expected. Constant magnitude bias, on the other hand, induces a roughly 
linear bias in the cosmological parameter fits. Qualitative confirmation of this can be 
found from the points of systematic-statistic equality as given in Table [TJ An increased 
magnitude dispersion does not cause any systematic shift in wq or w a . However, a 
dispersion of 0.40, 0.29 or 0.41 magnitudes added to the low, intermediate, or high 
redshift bin, respectively, doubles the size of the confidence regions. In a similar way a 
constant magnitude bias of roughly AM = 0.05 (slightly depending on redshift range) 
shifts the position of the 95.4% confidence level contour so that the best fit corresponding 
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to unbiased data lies outside the contour. 

The direction of the resulting bias in the (w , u> a )-plane depends on the redshift of 
the biased supernovae. A constant magnitude bias in the intermediate-z bin gives rise 
to a bias roughly along the major axis of the confidence level contour in the (w ,w a )- 
plane. A magnitude bias affecting only the high-z bin, on the other hand, induces a shift 
roughly along the direction perpendicular to the intermediate-z case, i.e. along the semi- 
major axis of the confidence level contour. The cosmological results are consequently 
more sensitive to a constant magnitude bias at high-Jj] than at intermediate-z. 

Similar trends can be seen in the (Qm, w)-plane, although the fits are roughly twice 
as stable considering systematic magnitude modifications (a bias of rougly AM = 0.1 
mag would reach statistical-systematic equality). 

Notice that these runs do not include the effects on color determination that 
magnitude errors could create (see §5.31 on color effects below). 

5.2. Redshift uncertainties 

The uncertainty in redshift propagates mainly in four ways into the estimation of 
cosmological parameters: 

(i) The lightcurve shape parameter used in the width-brightness relation, e.g. the 
"stretch" factor s (see [22]) is degenerate with the redshift as the latter is obtained 
from a fit of s(l + z) to the light curve. An error in (1 + z) is thus compensated by 
a corresponding error in s. 

(ii) The K-corrections applied correspond to the wrong redshift. 

(iii) The estimate of the color excess is biased. 

(iv) The points are offset in the horizontal axis of the Hubble diagram. 

Points (i) and (iv) can be studied either separately or together, results quoted here 
include effects (ii-iv) combined: no specific light curve shape dependent modifications 
were included in the current version of the code, but this is a simple extension planned 
for future studies since 5z = 5s. 

Results As long as spectroscopic redshifts are available either from the host galaxy 
or the SN la, the redshift uncertainty would typically only affect the second or third 
decimal place of the measured redshift. This is the case for most if not all the SNe la in 
the Gold sample. We note that a bias in the redshifts of the low-z (int-z) bin at the level 
of Az = 0.001 (Az = 0.007) would suffice to exceed the statitsical uncertainty (when 
considering the Wq — w a paramerization). Thus, this source of error is not negligible, 
although it is unlikely to be a major concern. However, future large-scale surveys may 
have to rely upon photometric redshifts for a large fraction of the supernovae. Studies 

f It might be of interest to note that adding a high-z bias of AM = 0.1 mag brings the best fit of the 
Gold data-set from a slight "offset" to quite precisely that corresponding to a cosmological constant, 
(w ,W a ) = (-1,0). 
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of photometric redshifts [2"3"l I2"H [25] have shown that the uncertainties are proportional 
to (1 + z). We model the size of the photometric redshift errors using the formula 
Sz = k(l + z), where the size of the uncertainty is described by the parameter k. We 
have studied the increased uncertainty due to this fact, when applied to all supernovae, 
for several different values of k. Photometric redshift uncertainties can give rise to large 
effects. Notable is that Gaussian uncertainties can give rise to significant bias. For the 
whole sample, the point of systematic-statistic equality is reached when the photometric 
redshift error is 5z = 0.006(1 + z) (see Table [1]). Since d! L changes rather slowly at high 
redshift, we expect the cosmo logical results to be more sensitive to redshift uncertainties 
at low and intermediate redshift than at high redshift. This is in agreement with what 
we find. If only the low redshift bin is affected by photometric redshift errors, the 
systematic errors starts to dominate at k = 0.008. Since this value of k is only slightly 
larger than what is needed for the whole sample, we conclude that spectroscopic redshifts 
are particularly important for low redshift SNe la. The same general trend can be seen 
in the VLm — w parameterization. 

5.3. Systematic effects on color 

Any bias or error in measurements of apparent magnitudes will propagate to the color 
determination and hence affect corrections for extinction. Biased magnitudes due to 
miscalibration between different filters could hence result in color errors. Use of multi- 
band color estimations would help stabilizing color systematics, to some extent limiting 
these effects. Both constant offsets and Gaussian errors in the color excess, E(B — V), 
were considered as systematic effects. 

Results Estimation of cosmological parameters is very sensitive to errors in colors: 
Already a bias of 0.02 in E(B — V) applied to any of the redshift bins would create 
a significant bias in the (wq, w a )-plane. For a significant bias in the (Om, to)-plane, a 
bias of 0.03 is sufficient. Since E(B — V) is a differential calculated from two (or more) 
filters this shows the extreme care necessary in photometry and filter calibration. The 
intrinsic colors of SNe la are poorly known, especially at high redshift. Thus, we rank 
this source of error as a critical one. 

5.4- Rv systematic effects 

Rv values, being the reddening due to interstellar dust in the host galaxy or a 
combination of effects including SN la physics and/or circumstellar dust, are notoriously 
hard to determine. We have studied how much offsets or random fluctuations from a 
fixed Milky Way-like value of Ry = 3.1 affect confidence level contours. The results 
indicate the systematic effects which arise from using Ry = 3.1 in the calculations while 
the true value is different. If the average value deviates significantly from current Milky 
Way estimates, a systematic bias will occur in any cosmology fit. 
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In this case, the systematic effect was introduced for all SNe la (allowing for the 
fact that virtually no host galaxy Ry's are known) as well as to the members of each 
bin separately. 

As an example of how systematic effects influence the scatter in the Hubble diagram, 
consider a bias in Ry for high-z SN la. The rms dispersion around the best fit Hubble 
diagram for different Ry values is shown in the left panel in Figure [6j The right panel 
in Figure [6] shows the dispersion in the Hubble diagram for different values of the 
dispersion, <jr v , in Ry. From the figure we conclude that there is no dramatic increase 
in the Hubble diagram dispersion for the range of Ry and or v considered^ Other 
systematics yield similar results. 




Kl 2 3 4 5 K 0.5 1 1.5 2 



Figure 6. Dispersion (rms) in the Hubble diagram due to uncertainties in Ry for high 
redshift SNe la. The right panel shows how the dispersion changes for different values 
of Ry. The left panel shows the effect of increasing the Gaussian dispersion in Ry. 



Results As can be seen from Table [T] a Gaussian with fairly large width (ctr v = 1.13) 
can be added to all SNe la before reaching the current statistical limit even for wq — w a 
fits. 

But this assumes an Ry distribution which is both centered on 3.1 and Gaussian. 
A non-Gaussian distribution could for example arise from different dust properties 
in different galaxy types. Simulations of systematic effects due to non-Gaussian 
distributions is another possible future application of SMOCK. An offset in the assumed 
value of total to selective extinction coefficient of ARy = 2 would be required to 
significantely bias the results. We note that this is at the level of discrepancy between 
the Galactic value and what has been derived from SNe la with the SALT technique 
[291 EH]- A linear evolution in redshift dRy/dz ~ 1.5 (Ry evolution in Tabled]) would 
be required to exceed the statistical uncertainty in the Gold data-set. 

| A close inspection of Figure [6] reveals a slightly smaller dispersion if a small negative R v bias is 
added. This shows that care is needed when minimizing hubble diagram residuals since a systematic 
bias can be introduced. 
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The importance of SN la colors can be seen here: For the intermediate redshift 
bin, the current data-set have average colors close to zero making this redshift bin less 
sensitive to -Ry-bias. This shows the advantage of low extinction supernovae and in 
general the promise of how future subsets of SNe la could be used to limit systematics. 
Obviously, actual knowledge of host galaxy dust properties would be preferable. 

5.5. Malmquist bias 

An effect corresponding to Malmquist bias, the systematic failure to detect the faintest 
elements of a population, was implemented through the option to, in a specific redshift 
interval, remove a certain fraction of the faintest SNe la and replace them with a (for 
one SMOCK iteration only) random copy of the same number of other SN la in the 
same redshift interval Jjj 

Results Adding a synthetic Malmquist bias is more natural for a completely simulated 
data-set, but runs using the Gold-set does give some indication of the size of the effect. 

Results were quite stable to Malmquist bias added to the data in the low and 
intermediate redshift bins. A significant bias did arise when Malmquist bias was added 
to the high-z bin, but in order for the effect to dominate over the statistical uncertainty a 
Malmquist ratio (the fraction of elements removed) of about 11% is required. This would 
need to be a systematic effect affecting all SNe la above z = 0.7. In the (£Im, w)-plane 
the 25% faintest SNe la would have to be missed by a survey to reach the statistical 
error limit. 

5.6. Gravitational lensing 

Gravitational lensing due to matter along the line of sight to a supernova, could give 
rise to amplification or de-amplification of the observed flux. The effect of gravitational 
lensing can be described by the magnification factor fi. Due to flux conservation 
the average flux from a large number of standard candles is the same as the flux 
from a standard candle not affected by lensing. The average magnification factor 
is consequently (//) = 1. A standard candle brighter or fainter than average would 
be described by /x > 1 and \i < 1, respectively. Gravitational lensing increases the 
dispersion of standard candles and could give rise to a selection bias, since amplified 
SN la are easier to detect than de-amplified ones. The more distant the standard candle, 
the more likely it is to be affected by gravitational lensing since the light emitted from 
it traverses more matter. When gravitational lensing is included, all SN la magnitudes 
are modified by a term — 2.51og 10 /i, where the magnification factors are drawn from 
appropriate redshift dependent probability distribution functions generated using the 
SNOC package [Hj. 

§ For a real data-set it might be more relevent to apply a "reversed" Malmquist effect, by removing a 
negative fraction of dim SNe la. This will with the current SMOCK-code add new dim objects. 
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Results Added lensing uncertainties does not significantly change the cosmological 
results, in accordance with earlier estimates [26J. This is an example of how detailed 
studies of possible systematics can lead to a situation where a systematic uncertainty 
can be said to be under control. 

5. 7. SN la spectral templates 

We have compared the results obtained using different spectral templates when 
calculating the K-corrections. The data were refitted using both our standard modified 
Nugent template [27J EE] and two other publicly available templates [301 IS]- This 
refitting procedure includes new extinction estimates and new K-correction calculations. 
The effects of different templates were studied using two modes: (i) Either the deviation 
of magnitude at epoch (the date of maximum I?-band brightness) due to template 
differences from our standard template was used or (ii) the maximal deviation at either 
epoch —10, or +10 days was taken as the systematic error. 

Results Since the different spectral templates are often used together with various light- 
curve fitting techniques, the comparisons we report here are somewhat oversimplified. 

Template differences using only epoch values, mode (i), was found not to be 
dominating. But in mode (ii) serious systematic shifts do occur. The cosmological 
fits performed using the maximal difference between the templates, roughly differ by 
2a. While not being a totally realistic comparison, it shows the key importance of 
using accurate SN la templates. These plots are available through the webpage and the 
SMOCK Gui (see AO below) . 

5. 8. SN la brightness evolution 

It is quite likely that the luminosity of the supernova is related to the properties of 
the white dwarf progenitor star, its companion and details of the propagation of the 
explosion. On the progenitor side, key parameters are the mass of the exploding white 
dwarf, the relative amounts of carbon and oxygen and metallicity. Thus, evolution in 
any of these parameters is a source of concern as it would lead to a drift in the intrinsic 
luminosity used to derive distances. In order to mimic an evolution of this kind, the 
effects of a redshift dependent bias were studied. This bias was described by a simple 
linear function of redshift, AM = kz, where the slope k can be positive or negative. 

Results Positive or negative evolution of intrinsic SN la brightness with redshift 
produce mainly bias effects. In general, an evolution of \k\ ~ 0.1 is needed to cause 
significant bias. We show the effects of evolution in the (Om, io)-plane in Figure HI 
where a clear bias in both Qm and w can be seen (as expected, the relative area stays 
the same). 
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6. Discussion 

Concerning our present knowledge of dark energy parameters, it is plausible that 
current data sets of SNe la are plagued by significant systematic uncertainties, such as 
cross-calibration errors, spectral template flaws, and uncertain extinction corrections. 
Systematic uncertainties may already dominate the error budget and will certainly do 
so in the future. To investigate, understand, and ultimately reduce or correct for these 
sources of error are hence of uttermost importance for the success of future precision 
cosmology. 

The purpose of this paper is twofold: to present a method to investigate the impact 
on the estimation of cosmological parameters due to a number of systematic effects 
and, as an example, investigate the potential impact of systematic uncertainties in the 
cosmology fits of the Gold data-set of Riess et al 2007. 

The SMOCK method, and all results for the Gold sample presented here, are 
somewhat dependent on the actual redshift binning used for the investigation and 
the details of the data set. Future improvements would include some form of 
parameterization of these effects as well. 

In general, any future reported deviation from a ACDM-cosmology would need to 
be supplemented with excellent control of all systematics discussed here, since most of 
them could account for such a result. 

6.1. Nightmare scenarios 

Various "nightmare" scenarios can easily be constructed out of the systematic effects 
considered here. Miscalibration between low and high-z data and biased colors would 
lead to large systematic effects. 

For example, any individual application of either an offset in inrinsic colors 
A(B — V) ~ 0.02, a linear evolution of the lightcurve shape-brightness corrected peak 
brightness of SNe la exceeding AM = 0.1 • z or a mean value of Ry of 2, instead of 3.1, 
would be enough to breach the statistical limit for the Gold data set. 

Should any of these effects be combined or added to template uncertainties, 
systematic effects several times current statistical errrors could be generated. 

6.2. DETF 

Comparing any results presented here with the targets set by the Dark Energy Task 
Force (DETF) [33] for future ( "stage four" ) instrument targets, it is clear that statistical 
errors will decrease significantly (as 1/y/n where n is the number of SNe la). But 
systematic effects will not scale in the same way, meaning that the points of systematic- 
statistic equality for bias studies also will scale as 1/ y/n. Future surveys will thus be 
extremely sensitive to systematic effects, and any bias will most likely come to dominate 
over statistical errors. 
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Thus, to make use of data from fourth generation instruments, control over 
instrumentation and calibration issues must be excellent. At the same time we can 
hope for a significant increase in the knowledge of understanding of systematic effects 
of astrophysical origin (developments are not easy to predict and are often not included 
in instrumentation plans but might be essential for future surveys). 

6.3. SMOCK GUI 

In order to help visualize the effects on the estimation of cosmological parameters 
due to systematic uncertainties, we have developed a small graphical application. 
This Graphical User Interface (GUI) displays the results obtained by the simulations 
performed with SMOCK. The application combines all results obtained for a particular 
data-set and allows different combinations of errors to be tried out. This tool, 
together with some of the data obtained for the Gold data-set, is available at 
www.physto.se/~nordin/smock. 

It should be noticed that all errors were calculated individually. In particular, 
a combined error in redshift and color-determination often leads to larger errors 
(e.g. through increased K-correction errors) than those displayed by the GUI. 

7. Conclusions 

SMOCK, a statistical Monte Carlo simulation approach towards quantification of 
systematic effects is presented. This method can give concrete answers to how sensitive 
SN la samples are to a wide variety of different systematic effects. We can estimate both 
systematic and statistical uncertainties in a time efficient way. 

Current SN la data-sets are likely to be influenced by unknown systematics, 
which quite possibly already are larger than the statistical errors. In order for future 
SN la surveys to be successful, the propagated effects into the cosmolgical fits must 
be understood. With the methods presented here this will be possible through a 
process where we first define acceptable systematic uncertainties to achieve some target 
confidence level in e.g. the impact on the derived dark energy properties. The SMOCK 
technique can be used to further define the instrumental requirements necessary to 
detect and possibly correct for any putative source of systematic uncertainty. 

Studies of a range of different systematics were performed on the Gold data- 
set, demonstrating the possibility of quantifying sensitivity to systematic effects. The 
importance of excluding the effects of a possible Ry or color bias and obtaining a good 
spectral template were highlighted. For increased precision future surveys will need to 
demonstrate that maximal systematic color effects are at the 0.01 magnitude level and 
that host galaxy dust properties do not have any systematic shifts over the AR V = 1 
level (or that possible such shifts are under control). 
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